System and method for use of a non-blocking process with a resource pool in a computing environment

ABSTRACT

In accordance with an embodiment, described herein is a system and method for use of a non-blocking process with a resource pool, for example a connection pool, or another type of computing environment. A generic wait-free approach can be used with a variety of different types of resource pools and pool elements. Threads which require access to a collection of resources do not have to wait for a lock on that collection of resources to become available. Although a borrowing thread may still be required to wait for a particular resource to be released, the wait-free approach allows the thread to browse through a list of available resources. The approach can be used with connection pools, or with other types of computing environments, to provide improved scalability in such systems.

CLAIM OF PRIORITY

This application claims the benefit of priority to U.S. ProvisionalPatent Application titled “SYSTEM AND METHOD FOR USE OF A NON-BLOCKINGPROCESS WITH A CONNECTION POOL OR OTHER COMPUTING ENVIRONMENT”,Application No. 62/194,734, filed Jul. 20, 2015, which is hereinincorporated by reference.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever.

Field of Invention

Embodiments of the invention are generally related to computerprocessing, and are particularly related to a system and method for useof a non-blocking process with a resource pool, for example a connectionpool, or another type of computing environment.

Background

In many computing environments, resource pools can be used to provideaccess to resources which merit caching, since recreating thoseresources would be computationally expensive. Some examples of resourcepools, and their pool elements, include database connections, memorybuffers, or sections of memory that are computationally expensive toallocate.

For example, in an application server or other computing environmentthat includes a database connection pool, such as an Oracle UniversalConnection Pool (UCP) environment, the connection pool acts as aresource pool that caches database connections, with each connectionpool element associated with a corresponding software object thatdefines a connection. A software application can request a particulartype of connection, and, if the connection pool includes a connectionthat can satisfy the request, that connection can be provided to theapplication in the form of access to the appropriate software object.The application can use the connection to perform some work, and thenreturn it to the connection pool, so that it can be made available forsubsequent requests, from the same or another application.

However, while some environments can synchronize access to a cache ofmany connections (for example, a first thread A may release a connectionto the pool, while another thread B searches for a best connection inthe pool), such model generally does not scale well. With softwareapplications increasingly being deployed to larger enterprise or cloudenvironments, such systems can benefit in being able to scale, tosupport much larger numbers of threads.

SUMMARY

In accordance with an embodiment, described herein is a system andmethod for use of a non-blocking process with a resource pool, forexample a connection pool, or another type of computing environment. Ageneric wait-free approach can be used with a variety of different typesof resource pools and pool elements. Threads which require access to acollection of resources do not have to wait for a lock on thatcollection of resources to become available. Although a borrowing threadmay still be required to wait for a particular resource to be released,the wait-free approach allows the thread to browse through a list ofavailable resources. The approach can be used with connection pools, orwith other types of computing environments, to provide improvedscalability in such systems.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates use of a non-blocking process, in accordance with anembodiment.

FIG. 2 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 3 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 4 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 5 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 6 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 7 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 8 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 9 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 10 further illustrates use of a non-blocking process, in accordancewith an embodiment.

FIG. 11 illustrates a flowchart describing a non-blocking process, inaccordance with an embodiment.

FIG. 12 further illustrates a flowchart describing a non-blockingprocess, in accordance with an embodiment.

FIG. 13 illustrates an exemplary use of a non-blocking process with aconnection pool environment, in accordance with an embodiment.

FIG. 14 further illustrates an exemplary use of a non-blocking processwith a connection pool environment, in accordance with an embodiment.

FIG. 15 illustrates the performance of an exemplary non-blocking processin a connection pool environment, in accordance with an embodiment.

FIG. 16 illustrates an exemplary non-blocking process, for use in aconnection pool environment, in accordance with an embodiment.

DETAILED DESCRIPTION

As described above, in many computing environments, resource pools canbe used to provide access to resources which merit caching, sincerecreating those resources would be computationally expensive. Someexamples of resource pools, and pool elements, include databaseconnections, memory buffers, or sections of memory that arecomputationally expensive to allocate.

In accordance with an embodiment, described herein is a system andmethod for use of a non-blocking process with a resource pool, forexample a connection pool, or another type of computing environment. Ageneric wait-free approach can be used with a variety of different typesof resource pools and pool elements. Threads which require access to acollection of resources do not have to wait for a lock on thatcollection of resources to become available. Although a borrowing threadmay still be required to wait for a particular resource to be released,the wait-free approach allows the thread to browse through a list ofavailable resources. The approach can be used with connection pools, orwith other types of computing environments, to provide improvedscalability in such systems.

For example, in accordance with an embodiment, the system enables a poolelement that is associated with a data or other resource, to be reservedin a reliable manner, such that only one thread has entire access tothat particular element for the duration of the reservation. During thereservation, other threads will be prevented from accessing thatparticular pool element. However, the reservation of a particular poolelement by a thread, for the purpose of determining the suitability ofits associated data or other resource, does not prevent those otherthreads from navigating through the remainder of the pool, potentiallyreserving and examining other pool elements.

Introduction to Resource Pools

In accordance with an embodiment, a pool can be viewed as a concurrentprogrammatic logic that manages pool elements and supports basicoperations such as: insert a pool element into the pool; remove a poolelement from the pool; and iterate through the pool seeking a specificpool element. Pools can be used to manage access to data or other typesof resources, in which case they are generally referred to herein asresource pools.

As described above, some examples of resource pool elements includedatabase connections, memory buffers, or sections of memory that arecomputationally expensive to allocate. However, a resource pool can alsobe generalized, such that any object of any class can be considered apool element.

Although some aspects of this functionality can be implemented usingcritical sections in the software application code, such an approach canlead to performance and scalability limitations. A critical section, forexample one which finally translates into monitorenter and monitorexitJava virtual machine (JVM) pseudo code instructions, forces threads towait to enter a monitor process, where those waiting threads are subjectto thread re-scheduling and context-switching. Additionally, thecritical section generally prevents other threads from performing workwhile one thread is within the critical section, which results in poorscaling performance when greater number of threads require access to thepool.

For example, in a connection pool environment that provides access to aconnection pool or cache using a critical section, when a threadrequests a connection, the connection pool or cache is generally locked,to find a best connection and return it, while other threads that mayalso want a connection are forced to wait.

As another example, a procedure to increment a counter may be configuredto get a particular value, add “1” to it, and store the new value. Sincemultiple threads accessing the counter at the same time could causecorruption, the counter can be provided as a critical section, so thatonly a single thread can enter it at a time, to access the blocked data,while other threads are placed in a queue waiting. Context-switching canbe used to idle and wake threads as they enter the critical section,which effectively operates as an atomic action by using severalnonatomic actions.

The need for such blocking, queuing, and context-switching means thatcritical sections are often large, computationally expensive,operations. When a critical section is used in a multiprocessorenvironment, since only one thread is able to enter the critical sectionat a time, this means that other processors may remain idle while theirthreads are being held.

An alternative approach to the use of critical sections is to provide await-free, or lock-free, approach using a hardware operation such ascompare-and-swap. In this approach, for example, a first thread can reada counter, increment it, and then perform a single hardware operation,which directs that, if the counter value is “X”, then modify thecontent. Another thread can attempt the same operation, in such a mannerthat the winning thread wins, and the losing thread performs a re-read.

Since the above approach is performed as a single machine instruction,the likelihood of overlap is small. The threads can perform awhile-loop, and spin while attempting to increment the counter; althoughin practice, with small operations, such operations do not spin verylong. Since there is no queue and no context-switch, the timing is smallcompared with the hundreds of lines of code in a typical criticalsection.

Non-Blocking Process

As described above, with software applications increasingly beingdeployed to larger enterprise or cloud environments, such systems canbenefit in being able to scale to support much larger numbers (e.g.,thousands) of threads, such as, in the example of a connection pool, theability to handle concurrent access to the connection cache by severalthreads.

In accordance with an embodiment, an additional pool operation issupported that allows a borrowing thread to reserve a pool element, sothat the thread can inspect that pool element to make a determination asto whether the pool element should be subsequently borrowed.

As described herein, in accordance with an embodiment, the action of“reserving” a pool element generally means “reserve to inspect” theelement, such that a thread reserves and inspects a particular element,because it is considering whether to borrow or otherwise use thatparticular element. Generally, a thread will reserve a pool elementduring a very short period of time. If the thread then determines toborrow the pool element, it will remove that element from the pool ofavailable elements. The borrowed element leaves an empty cell in theresource pool, which can be used if necessary during a subsequent poolinsert operation.

Additionally, in accordance with an embodiment, the resource pool islinearly-organized, including that the pool element reservation order isgenerally provided on a first-come-first-served basis for participatingthreads; threads are never blocked by reserved pool elements; and aparticipating thread has the option to revisit a reserved pool elementwhen that pool element is released.

The described approach provides for reliability of a pool element'sreservation, such that only a single thread will have entire access to aparticular pool element for the duration of its reservation. During theperiod or duration of reservation, other threads will be prevented fromaccessing that particular pool element. However, the reservation of aparticular pool element by a particular thread, for the purpose ofdetermining the suitability of its associated data or other resource,will not prevent those other threads from navigating through theremainder of the pool, potentially reserving and examining other poolelements.

In accordance with an embodiment, the described approach implements anon-blocking process including the use of atomic references withcompareAndSet operations, which can be translated into native hardwarecompare-and-swap spinlock instructions. This enables the system to avoidusing monitorenter and monitorexit pseudocode instructions; and insteaduse spinlocks which are atomic references in Java.

In accordance with an embodiment, an atomic reference can be provided bya class in the Java class library that implements an update operationusing compareAndSet. One means of accomplishing this is to set a valueif something is null. For example, with Java this can be indicated as“if field-equals-null then field equals new-value”. However, in amultithreaded environment, by the time a thread can complete thisoperation, another thread could have changed the value. To address,this, an atomic reference can be used to test equal-to-null and set to anew value that is performed at the hardware level, for example bydetermining a bit pattern that appears as null, and setting it to a bitpattern of the new value.

FIGS. 1-12 illustrate use of a non-blocking process, in accordance withan embodiment.

As illustrated in FIG. 1, in accordance with an embodiment, a system caninclude or provide a non-blocking pool logic 102, which when executed bya computer enables access to a resource pool or other list of poolelements 104, via a non-blocking pool application program interface(API) 106. The non-blocking pool logic also supports the use of aniterator queue 108, as further described below.

In accordance with an embodiment, each pool element within the list ofpool elements can be associated with a chunk or region of memory, andcontain a reference to a next pool element, a flag to indicate that thepool element is reserved (for example, by containing either a null, or areference to an iterator that has currently reserved this pool element),and a data block or field that points to a data or other resource to becached, for example a connection or other software object.

For example, in a connection pool environment, each pool element can actas a holder for a particular type of connection, which in turn can beused to perform some work, e.g., at a database.

As illustrated in FIG. 2, a pool element A 110, which is inserted orotherwise added to the resource pool, can be initially provided as afree or empty pool element, such that the element is not currentlyassociated with a particular data or other resource, nor is the elementcurrently reserved by an application thread.

As illustrated in FIG. 3, additional pool elements including, forexample, pool element B 120, can be similarly inserted or otherwiseadded to the resource pool, and can be similarly initially provided asfree or empty pool elements.

In accordance with an embodiment, the list of pool elements within theresource pool can have a list head, and a list tail. When a threadassociated with a calling application, for example one of applicationthreads A 114 or B 116, each of which are associated with their ownrevisit queues 115, 117 respectively, wants to insert a pool elementinto the list, using one or more operations 130 at the pool's API, thenon-blocking logic can parse through the one-directional list of poolelements to determine an appropriate insertion point.

In accordance with an embodiment, optionally a fill reference (fillRef)132 can be used to point to the first vacant pool element (i.e., with adata field=null) in the list. Pool elements are generally not removedfrom the list, which is instead extended as necessary, and can bereserved on a one-by-one basis: if a pool element is already reserved,then it can be ignored, otherwise for a pool element to be reserved, thethread can examine the pool element's data field, and use it bymodifying it accordingly.

In accordance with an embodiment, the system can use atomic references,implemented as compareAndSet operations, to ensure that another threadcannot insert a new pool element into the list at the same time, orcompete to reserve the same pool element. In accordance with anembodiment, these atomic references can be translated into nativehardware compare-and-swap spinlock instructions.

As illustrated in FIG. 4, a calling thread can reserve a particular poolelement, in this example, pool element A, which can be flagged toindicate that the pool element is currently reserved.

In accordance with an embodiment, once a pool element is reserved, otherthreads can ignore that pool element during the reservation period. Whena pool element is reserved by a thread, this does not necessarily meanthat the element is checked-out or borrowed (for example, in the mannerthat a database connection might be checked-out or borrowed by a callingapplication, to perform some work at the database); but instead meansthat the thread is considering the pool element for its use, and may, ormay not, continue to check-out, borrow, or otherwise use it. Otherthreads can skip that pool element for now, but they may return laterand attempt to retrieve it.

In an exemplary embodiment, which supports the use of a connection poolto access database connections, during the borrowing of a connection,the system allows a thread to reserve a pool element, ensure the elementis not currently borrowed, mark the element as borrowed, and thenrelease the element, leaving it in the same resource pool. In such anenvironment, a pool element is removed from the pool only when thatelement's associated connection is closed. In this regard, thereservation of a pool element, e.g., one associated with a connection,is not the same as, for example, the checking out of that element'sconnection.

For example, as illustrated in FIG. 5, the pool element A can bereserved 134 by (some) thread, in order for that thread to examinewhether the data or resource (e.g., connection object) 136 associatedwith that pool element, is potentially suitable for use by that thread,even though the underlying data or resource, in this example aconnection, may ultimately not be checked-out by the thread.

As illustrated in FIG. 6, if the pool element is suitable for furtheruse by a reserving thread, then its data field can be updatedaccordingly, to reflect that it is being checked-out, borrowed, orotherwise being used by a particular thread, effectively removing thatpool element from being made available within the resource pool.

In accordance with an embodiment, a thread reserves a pool element(e.g., one that may be associated with a particular connection) for avery short period of time, so it can check to see if it wants the dataor resource associated with that pool element (e.g., the connection). Inaccordance with an embodiment, the action of reserving a pool element isan atomic operation and is very quick, which also allows a thread toquickly un-reserve it, and move on to consider a next pool element(e.g., a next connection), as necessary.

An advantage of this approach includes that, if a thread fails toreserve a pool element, the thread can simply note that it has notlooked at that pool element, but can otherwise continue to performuseful work to determine if there is, perhaps, another pool element(e.g., connection) that may be suitable or satisfactory. Since there isno holding of threads, which continue to perform useful work, thedescribed approach provides improved performance over the use of, forexample, critical sections.

In accordance with an embodiment, a pool iteration operation is similarto that of insertion. To avoid having to parse through the wholeresource pool, the fill reference can be optionally used as a clue todetermine where to start parsing the list. Generally, the fill referenceis used only for insertion once it points out for a vacant slot forinsertion. In this sense, the fill reference operates as an optimizationin providing a reference point to start an iteration with, but is not arequired feature. For example, to perform a pool iteration, withoutusing a fill reference, the system can simply start at the head(listhead) and parse through pool elements one by one, to the tail(listtail), ignoring empty slots. Other threads that cannot access apool element at the same time can skip over that pool element, and makea note to revisit that pool element later if appropriate. When thesystem reaches the end of the list for a particular thread, it will havea list of any pool elements that particular thread had reserved, andpotentially other pool elements which that particular thread had skippedsince another thread had already reserved them.

In accordance with an embodiment, the above-described approach toiteration provides several benefits. For example, in some use cases, itmay not be necessary for a thread to reserve a specific pool element,the thread may just be seeking something that is “good enough”. In suchuse cases, a wait timeout can also be used to support e.g., thosesituations in which the system will give up waiting and return an error.

Additionally, the approach avoids the problem of thread starvation insituations having a lesser amount of resources. The iterator queueprovides a way to access skipped pool elements in a fair way, sincethese pool elements are subject to being revisited, and many threads maywant to revisit the same skipped pool element. Since thread schedulingis stochastic, and a thread might otherwise continually see these poolelements as reserved, that thread's progress could be retarded in somemanner. In accordance with an embodiment, to avoid such threadstarvation, the system forces threads to revisit in the order in whichthey first attempted to revisit the object, in a first-in-first-outmanner.

For example, as illustrated in FIG. 7, in accordance with an embodiment,the list of pool elements includes pool elements A and B as before, andnow also includes pool element C 140, of which pool element A hasalready been reserved for use with a data or other resource 162.

As such, in this example, if an application thread A requires a poolelement for use with a particular data or other resource (e.g., aconnection object) 164, then pool element A is skipped, and the nextpool element B is reserved 165, for use with that data or otherresource. The skipped pool element is added to the revisit queue 166,for the requesting thread, and the thread is added 168 to the iteratorqueue, which in this example includes an indication of the iterators forpool element A 172, and those for pool element B 174. The fillreference, if used, is optionally updated to point to the next free poolelement, in this example pool element C.

As illustrated in FIG. 8, which further illustrates the exampledescribed above, pool element C has now also been reserved for use witha resource 182. If an application thread B requires a pool element foruse with a data or other resource, then pool element C is skipped, andanother pool element D 150 is created, for use with that data orresource. The skipped pool element is similarly added to the revisitqueue 184, for the requesting thread, and the thread is added 185 to theiterator queue, which in this example now includes an indication of theiterators for pool element C 186.

As illustrated in FIG. 9, the new pool element D can be reserved 188 foruse with the data or other resource 189. The fill reference, if used, isagain updated to point to the next free pool element, in this examplepool element E 160.

As illustrated in FIG. 10, which further illustrates the exampledescribed above, in accordance with an embodiment, as the list of poolelements becomes increasingly reserved, for use with additional data orother resources 195, then additional pool elements, such as pool elementN 198 can be added, with the fill reference again optionally beingupdated, this time to the first free pool element in the list, which inthis example is pool element A. In this example, each of elements B, D,and E, are currently associated with a data or other resource, for useby a thread, and are not currently available within the resource pool.Elements A, C, and N are neither associated with a data or otherresource, nor are currently reserved by a thread; and are available forsubsequent pool insertions.

The above approach works to cache pool elements that have specialattributes, so can be useful in various use cases other than simplyreturning a first-available pool element.

For example, in a connection pool environment, a pool element can beassociated with a connection label, and an application thread may wantto find a particular pool element with a particular label—the poolelements themselves may all be different from one another. Additionally,as described above, in accordance with various embodiments, the systemcan support use cases in which a “good enough” match is desired, and notnecessarily the only match, or the best match. Additionally, the systemdoes not require a thread to wait for a best match if someone is usingit, instead it can proceed to look for another or good enough match.

Exemplary Non-Blocking Process

In accordance with an embodiment, an exemplary process or algorithm isdescribed hereunder, wherein the pool comprises a one-directional linkedlist of pool elements; and each pool element has its own iterator queue(another one-directional linked list containing references of threadsthat are going to revisit this pool element); and each participatingthread has its own separate queue of pool elements to be revisitedbecause they happened to be reserved during pool navigation by thisthread.

List of Pool Elements

In accordance with an embodiment, a one-directional linked list of poolelements can be used. Each pool element has two atomic references: areference for a next pool element (or null) and a reference to a threadthat has reserved this pool element at this time (or null, ifunreserved). The reference to an actual data (for example, a connectionobject or some other data or resource) is not atomic. As describedabove, optionally, a fill reference can be provided pointing to thefirst vacant pool element (data=null). Since the fill reference operatesmerely as an optimization hint, if it does not provide the correctreference, then this does not pose a problem, since in this case thewait-free pool can simply add a new slot in the end of a pool's list,instead of using an empty slot in the middle of the list. The fillreference is used and updated by insertion and removal operations.Initially, the fill reference points to the pool's list head.

In accordance with an embodiment, a pool element reservation operationchecks (with compareAndSet) for a reserved contents and, if it is null,writes a current thread reference in it.

Alternatively, a pool element release operation checks (withcompareAndSet) for a current thread reference, and if found, assigns anull. If a wrong thread reference has been found, then a failure israised, because a pool element has to be released by the same thread.

In accordance with an embodiment, the above pool element operations arenot included in the pool API as themselves, but are instead included aspart of the insertion and iteration operations.

Pool Element Insertion

In accordance with an embodiment, a pool element insertion operationdoes the following:

1. From the fillRef reference, a thread goes toward the list tail,trying to reserve each next pool element and if successful, checks for adata field. If the data is null, i.e., a pool element is empty, fills itup with new data and releases a pool element. The insertion is done.

2. While moving towards the list tail, a thread checks a next (withcompareAndSet) and if it null, assigns a newly created pool element withdata re-filled with the data to be inserted.

3. The fillRef reference gets updated accordingly.

Pool Iteration

In accordance with an embodiment, a pool iteration operation does thefollowing:

1. First pass: from the list head, go towards the list tail. Per eachiteration, an attempt to reserve a next pool element (if it exists,i.e., a thread did not reach the list tail) is performed. A thread canfail because this pool element is reserved by another thread, in whichcase this iterator enqueues a current pool element in its own revisitqueue, and enqueues itself into a queue of this pool element's iteratorqueue. If reserved, a pool element becomes accessible to a callingapplication. During the next move through a list, the current poolelement gets released, and the next one gets reserved.

2. Second and further passes: going through a revisit queue, an iteratortries all pool elements scheduled for a revisit, checking for twothings: whether the current iterator is on the head of the queue of pooliterators, and if a reserve operation is succeeded. Upon success, athread reserves a pool element and dequeues an iterator from aniterators queue. Upon a failure, this pool element stays in a revisitqueue for the next revisit pass, and an iterator stays in the iteratorqueue for this pool element.

Pool Element Removal

In accordance with an embodiment, a pool element removal operation canbe done only on a currently reserved operation. The element's data fieldis assigned to null. The fill reference gets updated accordingly.

Pool Iterators

In accordance with an embodiment, each pool iterator corresponds to aparticipating thread, plus some other useful information for iteratingthrough a pool, for example, a revisit queue.

Iterator Queue

In accordance with an embodiment, this is a one-directional list queuingiterators to revisit a pool element in a proper order.

An enqueue operation uses the compareAndSet operation for finding a listtail, and insert a new pool element on this tail. An operation can lookfor an appropriate iterator at the list head to revisit a pool element.

A dequeue operation removes an unneeded pool element from the list head.The dequeue operation does not need a spinlock operation on a head,because dequeue is performed by wiping out iterators from pool elementsand the pool elements themselves can be removed occasionally, withoutsynchronization.

Revisit Queue

In accordance with an embodiment, this is the thread local data,containing pool element to be revisited.

Exemplary Process Flows

FIG. 11 illustrates a flowchart describing a non-blocking process, inaccordance with an embodiment.

As illustrated in FIG. 11, in accordance with an embodiment, at step202, a non-blocking pool logic is provided, for use with a resourcepool, wherein the non-blocking pool logic enables access to a list ofpool elements, via a non-blocking pool application program interface(API).

At step 204, each pool element within the list of pool elements isassociated with a chunk or region of memory, wherein each pool elementcontains a reference to a next pool element, a flag to indicate that thepool element is reserved, and a data field that points to a data orother resource, such as, for example, a database connection or othersoftware object to be cached.

At step 206, upon receiving a request from an iterator thread associatedwith a revisit queue, to insert a pool element into the pool, using oneor more operations at the pool's API, the one-directional list of poolelements is parsed to determine an appropriate insertion point, andinsert or reserve an appropriate pool element to be used in accessing aparticular data or other resource.

FIG. 12 further illustrates a flowchart describing a non-blockingprocess, in accordance with an embodiment.

As illustrated in FIG. 12, in accordance with an embodiment, at step210, upon receiving a request from an iterator thread associated with arevisit queue, to access a pool element in the pool, using one or moreoperations at the pool's API, the one-directional list of pool elementsis parsed to reserve an appropriate pool element to be used in accessinga particular data or other resource.

At step 212, during iteration by the iterator thread, on a first pass,an attempt is made to one of reserve a particular pool element by thethread, or enqueue the pool element in the thread's own revisit queueand enqueue the thread into the pool element's iterator queue, so thatif reserved, the pool element becomes accessible to a callingapplication associated with the iterator thread.

At step 214, during iteration by the iterator thread, on second andsubsequent passes, the thread's own revisit queue is parsed, includingall pool elements scheduled for a revisit, to determine whether thecurrent iterator is at the head of the pool element's iterator queue,and if a reserve operation has succeeded.

At step 216, upon a success, the iterator thread reserves the particularpool element and dequeues an iterator from the pool element's iteratorqueue.

At step 218, upon a failure, the particular pool element remains in arevisit queue for subsequent revisit passes, and the iterator threadremains in the iterator queue for the pool element.

UCP Implementation

FIG. 13 illustrates an exemplary use of a non-blocking process with aconnection pool environment, in accordance with an embodiment.

As illustrated in FIG. 13, in accordance with an embodiment, anapplication server/database environment 230, which includes physicalcomputer resources 232 (e.g., processor/CPU, memory or networkcomponent), can include or provide access to a database 234. Aconnection pool logic 240, including a non-blocking pool logic 242, cancontrol 252 the creation and allocation of objects within a connectionpool 250, including connections that are currently in use 254, andconnections that are idle 256. Labeling connections allows a softwareapplication 260 to attach name/value pairs to a connection. The callingapplication can then request 262 a connection with a desired label fromthe connection pool. By associating labels with connection states, anapplication can retrieve an already-initialized connection from the pooland avoid the time and cost of re-initialization.

FIG. 14 further illustrates an exemplary use of a non-blocking processwith a connection pool environment, in accordance with an embodiment.

For example, the connection pool can include a plurality of connections,including connection A labeled (Blue) and connection B labeled (Green)that are currently in use, and connections C, D, E, F and N that arecurrently idle and which can be likewise labeled. If a softwareapplication wishes to make a request on the database, using a particulartype of connection, it can make a getConnection request 264, say for a(Red) connection. In response, the connection pool logic can eitherprovide an existing (Red) connection if one exists; create a newconnection X (Red); or repurpose an existing connection, e.g., E (Red),to connect 266 to the database.

FIG. 15 illustrates the performance of an exemplary non-blocking processin a connection pool environment, in accordance with an embodiment.

As illustrated in the example of FIG. 15, which illustrates aperformance chart 270 that compares borrow/return request performancefor a UCP environment which uses a non-blocking or wait-free process,versus an (“old” UCP) environment which does not use such a process, theamount of requests processed per second, as the number of concurrentthreads accessing a pool increases, is improved when using anon-blocking process such as that described herein.

In the example illustrated in FIG. 15, the number of concurrent threads(shown on the X-axis) are competing for, in this instance, 6 physicalconnections, to borrow and immediately return back to the pool. Thethreads are not performing any further tasks on each connection, theysimply borrow and return the connection. This example illustrates thespeed of the borrow/return cycle, rather than the work performed byconnections, as might be found in an environment in which a largeamounts of threads are competing for a very limited amount of resources.

FIG. 16 illustrates an exemplary non-blocking process, for use in aconnection pool environment, in accordance with an embodiment.

As illustrated in FIG. 16, in accordance with an embodiment, at step280, a non-blocking pool logic is provided for use with a resource pool,wherein the non-blocking pool logic enables access to a list of poolelements, via a non-blocking pool application program interface (API),including that the resource pool is a connection pool, and that each ofone or more pool elements are associated with connections which acalling application can use to connect with a database.

At step 282, upon receiving a request from an iterator thread associatedwith a revisit queue, to insert a pool element into the pool, using oneor more operations at the pool's API, the system parses through theone-directional list of pool elements to determine an appropriateinsertion point, and insert an appropriate pool element reserving anassociated data or other resource; and/or reserve a pool element so thatthe pool element becomes accessible to a calling application associatedwith the iterator thread.

At step 284, the inserted or reserved pool element reserving anassociated data or other resource is returned to the callingapplication, in the form of access to a connection object which thecalling application can use to connect with the database.

Embodiments of the present invention may be conveniently implementedusing one or more conventional general purpose or specialized digitalcomputer, computing device, machine, or microprocessor, including one ormore processors, memory and/or computer readable storage mediaprogrammed according to the teachings of the present disclosure.Appropriate software coding can readily be prepared by skilledprogrammers based on the teachings of the present disclosure, as will beapparent to those skilled in the software art.

In some embodiments, the present invention includes a computer programproduct which is a non-transitory storage medium or computer readablemedium (media) having instructions stored thereon/in which can be usedto program a computer to perform any of the processes of the presentinvention. Examples of the storage medium can include, but is notlimited to, any type of disk including floppy disks, optical discs, DVD,CD-ROMs, microdrive, and magneto-optical disks, ROMs, RAMs, EPROMs,EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards,nanosystems (including molecular memory ICs), or any type of media ordevice suitable for storing instructions and/or data.

The foregoing description of embodiments of the present invention hasbeen provided for the purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed. Many modifications and variations will be apparent tothe practitioner skilled in the art. For example, while several of theexamples described above illustrate the use of a non-blocking process ina connection pool environment, in accordance with various embodiments,the approach described herein can also be used with other types ofcomputing environment.

The embodiments were chosen and described in order to best explain theprinciples of the invention and its practical application, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with various modifications that are suited tothe particular use contemplated.

What is claimed is:
 1. A system for use of a non-blocking process in acomputing environment, comprising: one or more computers, including aprocessor and a non-blocking pool logic executing thereon, whichprovides access to a resource pool associated with a list of poolelements, by a plurality of threads, including enabling one or more poolelements to be selectively reserved, so that only a reserving thread hasaccess to a particular pool element for a duration of a reservation, andduring the reservation, preventing other threads of the plurality ofthreads from accessing the particular pool element, and allowing theother threads to navigate through a remainder of the pool elements,reserving and examining others of the pool elements; wherein each poolelement is associated with a data or other resource; and wherein each ofthe plurality of threads access the pool in a non-blocking manner toselectively reserve pool elements, for use by the thread in determiningwhether to use the data or other resource associated with those poolelements.
 2. The system of claim 1, wherein each pool element isassociated with a flag that indicates whether that pool element isreserved for use by a thread, and a pointer to the data or otherresource associated with that pool element, that is usable by a threadto access the data or other resource associated with that pool element.3. The system of claim 1, wherein the system uses atomic compare and setoperations to check availability of and to reserve pool elements for useby the threads.
 4. The system of claim 1, wherein during iteration ofthe resource pool by a thread to locate a desired pool element, if thedesired pool element is reserved by another thread, then the desiredpool element is added to a skipped list associated with that thread. 5.The system of claim 1, wherein the system is used in a connection poolenvironment, including that each pool element is associated with aconnection that can be reserved by a thread.
 6. The system of claim 1,wherein the particular pool element is reserved momentarily by a threadto perform a check to determine whether the pool element should beborrowed by the thread for further use.
 7. The system of claim 1,wherein the system is used in a connection pool environment, includingthat each pool element can be associated with a connection that can beused by a software application.
 8. A method for use of a non-blockingprocess in a computing environment, comprising: providing, at one ormore computers, access to a resource pool associated with a list of poolelements, by a plurality of threads, including enabling one or more poolelements to be selectively reserved, such that only a reserving threadhas access to a particular pool element for a duration of a reservation,and during the reservation, preventing other threads of the plurality ofthreads from accessing the particular pool element, and allowing theother threads to navigate through a remainder of the pool elements,reserving and examining others of the pool elements; wherein each poolelement is associated with a data or other resource; and wherein each ofthe plurality of threads access the pool in a non-blocking manner toselectively reserve pool elements, for use by the thread in determiningwhether to use the data or other resource associated with those poolelements.
 9. The method of claim 8, wherein each pool element isassociated with a flag that indicates whether that pool element isreserved for use by a thread, and a pointer to the data or otherresource associated with that pool element, that is usable by a threadto access the data or other resource associated with that pool element.10. The method of claim 8, further comprising using atomic compare andset operations to check availability of and to reserve pool elements foruse by the threads.
 11. The method of claim 8, wherein during iterationof the resource pool by a thread to locate a desired pool element, ifthe desired pool element is reserved by another thread, then the desiredpool element is added to a skipped list associated with that thread. 12.The method of claim 8, wherein the method is performed in a connectionpool environment, including that each pool element is associated with aconnection that can be reserved by a thread.
 13. The method of claim 8,wherein the particular pool element is reserved momentarily by a threadto perform a check to determine whether the pool element should beborrowed by the thread for further use.
 14. The method of claim 8,wherein the method is performed in a connection pool environment,including that each pool element can be associated with a connectionthat can be used by a software application.
 15. A non-transitorycomputer readable storage medium, including instructions stored thereonwhich when read and executed by one or more computers cause the one ormore computers to perform a method comprising: providing access to aresource pool associated with a list of pool elements, by a plurality ofthreads, including enabling one or more pool elements to be selectivelyreserved, such that only a reserving thread has access to a particularpool element for a duration of a reservation, and during thereservation, preventing other threads of the plurality of threads fromaccessing the particular pool element, and allowing the other threads tonavigate through a remainder of the pool elements, reserving andexamining others of the pool elements; wherein each pool element isassociated with a data or other resource; and wherein each of theplurality of threads access the pool in a non-blocking manner toselectively reserve pool elements, for use by the thread in determiningwhether to use the data or other resource associated with those poolelements.
 16. The non-transitory computer readable storage medium ofclaim 15, wherein each pool element is associated with a flag thatindicates whether that pool element is reserved for use by a thread, anda pointer to the data or other resource associated with that poolelement, that is usable by a thread to access the data or other resourceassociated with that pool element.
 17. The non-transitory computerreadable storage medium of claim 15, further comprising using atomiccompare and set operations to check availability of and to reserve poolelements for use by the threads.
 18. The non-transitory computerreadable storage medium of claim 15, wherein during iteration of theresource pool by a thread to locate a desired pool element, if thedesired pool element is reserved by another thread, then the desiredpool element is added to a skipped list associated with that thread. 19.The non-transitory computer readable storage medium of claim 15, whereinthe system is used in a connection pool environment, including that eachpool element is associated with a connection that can be reserved by athread.
 20. The non-transitory computer readable storage medium of claim15, wherein the particular pool element is reserved momentarily by athread to perform a check to determine whether the pool element shouldbe borrowed by the thread for further use.